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ABSTRACT 

A nonmetric multidimensional scaling technique was 
employed to determine the characteristics of the ideal professor- 
Subsequently, the results were compared across varying levels of 
program personnel, including administrators, faculty, staff and 
students. , Centered in a relatively self-contained academic program of 
a large midwestern university, this study used a free response method 
of data collection — a relatively unknown technique. Consistent 
differences were found among various levels of personnel as to their 
perception of the ideal faculty member. In addition, the procedure 
provided a means of comparing the perceived performance of individual 
faculty members. In general, the data seemec to indicate that 
subgroups of personnel were operating from what appeared to be a 
consistent frame of reference, but that they rationalized the 
characteristics somewhat differently. Moreover, the perceptions were 
not consistent among groups. These discrepancies might result in role 
expectations which might go unfulfilled or overfulfilled, causing 
probable miscommunication and lack of efficiency, affecting the 
morale of involved personnel. Internal conflict could also be a 
possibility. (MV) 
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IDENTIFYING THE CHARACTERISTICS OF THE IDEAL PROFESSOR: 
AN APPLICATION OF MULTIDIMENSIONAL SCALING^ 



William E, Loadman 
The Nisonger Center 
The Ohio State University 

The purpose of this inquiry was to determine the applicability of a 
non-metric multidimensional scaling technique in identifying the characteristics 
of the ideal professor. A subsequent purpose was to compare the results of the 
technique across varying levels of program personnel. Personnel levels included 
administrators, faculty, staff and students from a relatively self contained 
academic program in a large midwestem university, A free response method 
of data collection was employed- and this study presents the procedures and 
findings associated with the use of this relatively unknown technique. Consistent 
differences were found among various levels of personnel -as to their perception 
of the ideal and various faculty members. This procedure also demonstrated 
a means of comparing the perceived performance of individual faculty members* 

BACKGROUND 

A problem which consistently confronts university administrators and program 
personnel is identifying the criteria or characteristics which can be used as 
standards by which an individual's performance can be assessed. To date, many 
of these efforts have relied upon (1) a rational selection by program 
administrators, (2) a factor analytic study identifying relevant dimensions 
or characteristics, (3) peer evaluation or (4) implicit non-specific criteria 
(As tin and Lee, 1966; Cos tin, Greenough, and Menages, 1971; Wilson, Dienst, 
and Watson, 1973). Multidimensional scaling has been suggested as an alternative 
procedure for the identification of these characteristics. 



The author wishes to acknowledge the assistance of Ann Ruedele and 
Mary Kim for their .efforts in this study. 



Subkoviak and Levin (1974) used a non-metric multidimensional scaling procedure 
to ascertain the characteristics of an ideal professor using faculty members from 
a given university department. They found that the procedure could be invoked 
In a meaningful way and does discriminate among individuals. Kruskal (1964, 1969) 
developed a scaling procedure which utilizes ordinal level data based on multiple 
responses and reduces the input using a non-metric procedure to a number of 
dimensions. Shepard (1962), Carroll and Chang (1964), and Carroll (1972) indicate 
the feasibility of utilizing non-metric multidimensional scaling as a procedure for 
assessing distance among rated stimuli. 

Using Subkoviak and Levin's model of non-metric multidimensional scaling, 
this inquiry attempted to determine the characteristics of an ideal professor 
from the criteria generated by various levels of program personnel. It also 
examined the similarities and discrepancies of various levels of staff as to 
their perceptions! of individual faculty members with respect to these ideal 
characteristics. Finally, it attempted to replicate a study by Subkoviak and 
Levin (1974) which used only one level of respondents. 

METHODOLOGY 

The specific methodology used in this study can be separated into the 
categories of subjects, instrtimentation, data collection and data reduction; 
a discussion of each category follows. 
Subjects 

Each instrument was completed by 43 individuals from a specific, relatively 
self contained academic program in a large midwestern university. There were 
five administrators, fifteen faculty members, fifteen staff members and eight 
students completing the instrument. All individuals were associated with the 
program and familiar with the faculty members. 
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Instrumentation 

An open ended method of data collection was used in this study. An 
instrument was designed to allow each respondent to first list the characteristic 
important in describing the ideal professor and to subsequently define the 
relative importance of each by appropriately dividing the total of 100 points 
among the various characteristics, the largest number of points being assigned 
to the most important characteristic, etc. Each characteristic listed had to 
receive at least one point and the sum of points for a given respondent had to 
total 100^ 

Once thig ideal characteristics and concomitant weights were individually 
established, the respondents were instructed to use each characteristic which 
they identified and create a unidimensional scale on that characteristic with 
a minimum of two and maximum of five points on a continuum. The determination 
of the continuum was left to the discretion of the individual. For example, 
it might have been: high, medium, and low; good, bad; frequently, infrequently; 
high intensity, moderate intensity, low intensity, very low intensity; etc. 
This procedure was repeated for each characteristic identified by the individual 
respondent. 
Data Collection 

After individually creating the characteristics, weight, and concomitant 
unidimensional scales, the respondents were given an enumerated list of 26 
faculty members in the program and were instructed to sort the faculty members, 
one characteristic at a time, into the various categories on their scale. 
In addition, they were instructed to place the ideal professor Into the scale 
category which they thought was most appropriate. This process was repeated 
for each characteristic the respondent identified. 

■ . .5 
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Data Reduction 

Since each individual respondent identified a unique set of characteristics 
and a unique scale for each characteristic, it was necessary to use a methodologica 
procedure which would allow the data to be put in a common metric for comparative 
purposes. This was achieved through the application of a non-metric multi- 
dimensional scaling pz'ocedure (Kruskal and Carmone, 1969), One large advantage 
of the non-metric multidimensional scaling procedure is that the procedure has 
been demonstrated to function very adequately when the ntuober of respondents is 
equal to or less than the number of stimuli. Obviously the greater number of 
respondents the greater the stability of the procedure. 

Using a procedure similar to that used by Wish, Deutsch and Biener (1970) 
and Subkoviak and Levin (1974), similarity indices were constructed for all 
possible pairs of the 27 rated individuals (26 faculty members plus the ideal). 
Each similarity index ranged in value from .00, indicative of a low similarity 
rating, to 1,0, the highest possible similarity rating. Specifics of the index 
development are discussed below. 

One hundred percent of the instruments (43) were returned and for each of the 

k respondents (instruments), the measure of similarity between each pair (i and j) 

of the 26 faculty members and the ideal professor was computed. For a given 

respondent, the similarity index between faculty members i and j (summed across 

categories) is defined as: (S^^) =^ W_Cd/4 4) 

^ k c=l c 

the number of characteristics of the ideal professor 
determined by the respondent (k) 

weight (.01 to 1.0) determined by respondent (k) for 
characteristic C 

1 if faculty members 1 and j sorted into same scale 
category by respondent (k) with respect to characteristic C 
0 if faculty members i and j sorted into different 
scale categories by respondents (k) with respect to 
characteristic C 



where 
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An overall average measure of similarity was calculated for each pair of 
faculty members by summing the individual similarity measures across the k 
respondents and dividing the total by the number of respondents as follows: 



Twenty seven faculty pairs were rated which resulted in a 27 x 27 matrix of 
similarity values S^j. There are (N)(N-l)/2 = 27(27-l)/2 = 351 unique similarity 
values in the matrix. Using the similarity values matrix, ordinal levels of 
measurement, the dimensionality of the ideal professor characteristics was then 
extracted through the non-metric multidimensional scaling computer program 
(Kruskal and Carrao*:e, 1969). This procedure was accomplished using the data 
from all 43 respondents. A similar procedure was employed to ascertain the 
dimensionality of the ideal professor characteristics as identified by each 
subgroup of respondents, i.e., administrators, faculty, staff and students. 

RESULTS 

Preliminary analysis of all the data revealed the most salient characteristics 
listed by the respondents (see Table 1). Most prominent were instruction, 
service, interpersonal relations, personal characteristics and research 
orientation. Obviously the specific language of the characteristic differed 
from respondent to respondent. Since these characteristics were given numerical 
weight, it was possible to determine the importance the respondents attached to 
the various characteristics. Teaching received .53 of the emphasis; service 
received .20; personal qualities J.1; interpersonal relations ,09; and research 
skills .06. 
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Using the multidimGnsional scaling program, a series of iterations was 
performed on the data beginning with a five dimensional solution down to a one 
dimensional solution. The stress values were obtained and graphed to determine 
the best fit of the data; a noticeable "elbow" occurred at two dimensions 
which suggested a two dimensional solution to the problem. Using the two 
dimensional solution, the final configuration of points was graped for the 26 
faculty members and the ideal professor (see figure 1). The intercept of the 
two dimensions represented the perceived average of all individuals rated by the 
respondents. The graph represents the ranked distance (dj^j) between professors 
i and j« This distance corresponds to the similarity index S^^ but transformed 
Into a two dimensional array. 

The two dimensions were then labeled through a procedure similar to that 
used in naming factors in a factor analysis, i.e., one examines the variables 
loading heavily on the factor and rationally/ Intuitively labels the dimension. 
In this example, the variables are faculty members. The weights assigned by the 
respondents were heavily concentrated in the areas of instruction and service 
lending credence to the two dimensional solution and the naming of the dimensions. 
In addition, the two dimensions were also found in the Subkovlak and Levin (1974) 
study. 

The weights assigned by each subgroup in determining the relevant characteristics 
are presented in table 1. Visual inspection of the weights reveals specific 
discrepancies across categories of staff, but general consistency in identifying 
and weighting domains. 

The multidimensional scaling program was similarly applied to the data from ^ 
each subgroup. Examination of the stress values for administrators, ^faculty ^^\v 
and staff indicated a distinct elbow at two dimensions,: Pl^ 
dimensional arrays cons 




-/- 

for each group. The stress values obtained from the student data indicated 
a distinct elbow at four dimensions. The two primary dimensions were identified 
as instruction and service and for the sake of comparability across groups the two 
dimensional solution was substituted. 

The resulting two dimensional plot for each group can be found in figures 
2, 3, A and 5. These figures graphically display the relative positions of the 
27 professors as perceived by administrators, faculty, staff and students. 
Individual faculty members are consistently alphabetized across plots; the 
ideal professor is indicated by an asterisk, 

DISCUSSION 

Through -the use of this "opeil-ended" method of data collection and the 
application of non-metric multidimensional scaling, a unique characterization 
of the ideal professor was obtained. In addition, the positions of a specific 
set of faculty members were determined in relation to the ideal and each other; 
these projections were obtained from administrators, faculty, staff, students 
and a composite. Each plotting represents a direct comparison of each faculty 
member with each other faculty member and the ideal as perceived by the group 
of responders. 

Two specific dimensions were consistently identif ied--instruction and 
service--across all groups of respondents. Instruction receiv6d a very heavy 
emphasis and service received slightly less emphasis. Othpr characteristics 
of the ideal professor did not receive sufficient emphasis to allow for meaningful 
interpretation. 

As one examines the location of the ideal professor across groups, there is 
an interesting finding; the perception of the ideal differs considerably across- .^^ 
groups, even though there is basic agreement on the number of dimensions a^ 

■ .■- . , • . .■ .':■[ .-. 

characteristics on these dimensions. Note in particular the large discrepancy 
between the perceptions of the staff and those of administrators and faculty. /; 



Everyone agrees that the ideal is above average on the instructional dimension 
(in varying degrees) and yet the staff see the ideal being heavily involved 
in the service delivery activities while other groups see the ideal as being 
less involved in service delivery. 

Another interesting finding is that perceptions about individual faculty 
metabers relative to each other and the ideal differ considerably across groups, 
yet there is consistency within groups as indicated by the stress values. For 
example, individual W is placed in the upper right quadrant by administrators and 
staff (in markedly dissimilar places in the quadrant) the lower right quadrant 
by faculty and the lower left quadrant by students. On the other hand, individual 
Y is consistently placed close to the ideal for all groups except staff. 

In general, the data seem to indicate that subgroups of personnel are 
operating from what on the surface appears to be a consistent frame of reference, 
e.g., number of dimensions and characteristics of the dimensions, but they 
opera tionalize the characteristics somewhat differently. The perceptions of 
individuals and the ideal are not consistent across groups. These discrepancies 
may result in potential role expectations which may go unfulfilled or over 
fulfilled, as the case may be. Without resolution of these discrepancies, program 
personnel may be using the same language to convey different meanings. At a 
minimum, this will lead to miscommunication and probable lack of efficiency. 
It could also affect the morale of involved personnel. There is even the " 
possibility of internal conflict. 

These data confirmed two of the three dimensions identified by Subkoviak 
and Levin (1974) and Wilson, et. al. (1973) but did not elicit a significant 
third dimension (research) as found in the previous works. This is not surprising 
as the clearly delineated two fold thrust of the program under study is 
instruction with the delivery of service as the vehicle for the instruction. 



There is very little emphasis upon research in the program. If this study were 
conducted in other settings, the results of course may be different. For instance 
other organizations may be more research oriented so that. the identified 
characteristics would be more heavily oriented toward research and publications 
and/or other factors while characteristics as identified in this study might 
receive considerably less emphasis. 

This study has demonstrated the feasibility and applicability of using a 
non-metric multidimensional scaling technique to obtain responses to a set of 
stimuli using a series of non-common tools (characteristics and scales). It 
was illustrated that this procedure can be employed with diverse groups of 
respondents. Possible applications are as follows: (1) The results could 
be used to determine common criteria for input into personnel evaluation by 
group consensus. (2) By utilizing the common criteria program decision makers 
could modify individual behaviors to be in line with expectations or to modify 
the expectations to be in line with the individual's performance. (3) These 
data could potentially be used for feedback to the individual relative to 
other's perceptions of the individual's performance., (4) These data might 
potentially be used for salary negotiations, promotion, tenure, etc. (5) 
(5) Programmatic emphasis could be increased, modified, etc., based upon the 
outcome of these data. (6) These data could be used as an evaluative _ ^ 
device in identifying consistency/inconsistency of perceptions on programmatic 
functioning across levels. of staff. 



Table 1 



Characterization of the Ideal Professor 
and Corresponding Assigned Weights 



Domain 
Teaching 



Typical Chpracteirlstlcs 

Knowledge of subject matter 

Ability to communicate knowledge 

Good course organization 

Ability to Interest and motivate students 



Weight 



.53 



Service 



Service to clients 

Clinical skills 

Interdisciplinary skills 

Skillful and mature application of skills 



.20 



Personal Qualities 



Intelligence 
Dedication 
Integrity 
Maturity 



.11 



Interpersonal Relations 
Skills 



Interpersonal management skills 
Ability to relate to others 
Flexibility 
Approa chab i 1 i t y 



.09 



Research Skills Research .06 

Publication 

Generates research ideas 



12 



Table 2 



Weights Assigned to Ideal Professor 
Characteristics by Levels of Program Staff 



Domains Total Group Administrators Faculty Staff Students 



• 


N=43 


N=5 


N=15 . 


N=15 


N=8 


Teaching 


.53 


.59 


.52 


.55 


.48 


Service 


.20 


.16 


.16 


.21 


.24 


Interpersonnel 
Relations 


.11 


00 


.12 


.07 


.12 


Personal 
Qualities 


.09 


.12 


.15 


.09 


.06 


Research 


.06 


.07 


.05 


.06 


.08 
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Figure I 



Two Dimensional Array of Faculty and Ideal 
Professor Characterization as Perceived by Respondents 
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Figure 2 



Two Dimensional Array of Faculty and Ideal 
Professor Characterization as Perceived by Administrators 
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Figure 3 



Two Dimensional Array of Faculty and Ideal 



Professor Characterization 


as Perceived by Faculty 
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Two Dimensional Array of Faculty and Ideal 
Professor Characterization as Perceived by Staff 



Figure 5 



Two Dimensional Array of Faculty and Ideal 
Professor Characterization as Perceived by Students 

High- - Ins truction 



Service 
Low 



0« 



F» I • 



J • 



U , 



s* 



A- 



O 



High 



Low 



18 



ERIC 



BIBLIOGRAPHY 



Astin, A., Lee, C. "Current Practices in the Evaluation and Training of 
College Teachers." Education Record , 47(3), 1966. 

Carroll, J.D., Chang, J.J. "A general index of nonlinear correlation 
and its application to the interpretation of multidimensional 
scaling solutions. American Psychologist , Vol. 19, 1964. 

Carroll, J.D. Individual Differences and Multidimensional Scaling. In 
Shepard, Ronney and Nerlane (Eds) Multidimensional Scaling , Vol. 1. 
Theory New York. Seminar Press, 1972. 

Costin, F., Greenough, W. , Menages, R. "Student Ratings oi ollege 
Teaching: Reliability, Validity and Usefulness." Review of 
Educational Research . Vol. 41, 1971. 

Kruskal, J.B. "Multidimensional Scaling by Optimizing Goodness of Fit to. 
a Nonmetric Hypothesis," Psychometrika . 29, 1964. 

Kruskal, J.B. "Nonmetric Multidimensional Scaling: A Numerical Method." 
Psychometrika , 29, 1964. 

Kruskal, J.B., Carmone, F. "How to Use Multidimensional Scale and Other 
Useful Information." Unpublished paper, Bell Laboratories, 1969. 

Shepard, R. "The Analysis of Proximities: Multidimensional Scaling 

with an Unknown Distance Function. II" Psychometrika . 27, 1962. 

Subkoviak, M. , Levin, J. "Determining the Characteristics of the Ideal 
Professor: An Alternative Approach." Journal of Educational 
- Measurement . Vol. 11, No. 4, Winter 1974. 

Wilson, R., Dienst, E., Watson, N. "Characteristics of Effective 

College Teachers as Perceived by Their Colleagues." Journal of 
Educational Measurement . Vol. 10, 1973. 

Wish, M. , Deutsch, M. Biener, L. "Differences in Conceptual Structures of 
Nations: An Exploratory Study." Journal of Personality and 
Social Psychology . 16, 1973. 



19 



